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The purpose of this study was to determine the specific skills and competencies 
that area employers require in their entry-level employees and to identify specific 
student-produced performance products which applicants can use to signal proficiency in 
the respective skills and competencies. The study involved surveying a broad 
organizational and industrial cross section of Tempe, Arizona and vicinity employers. A 
modified Delphi technique of inquiry was used, employing three rounds of detailed 
questionnaires over a five month period surveying a panel of 24 hiring managers, human 
resources specialists, small business owners and cooperative occupational education 
specialists. 

With each survey round, both quantitative and qualitative data were gathered. 

The data were summarized and used to construct the next round survey questionnaire. 

The Secretary of Labor's Commission on Achieving Necessary Skills (SCANS) Report 
served as the basis for rating the importance of a variety of entry-level workplace skills 
and competencies. The panel of experts generated performance product ideas, which 
could signal proficiency in each of the SCANS skills and competencies. In subsequent 
rounds, the panel rated and ranked the performance products on Likert-type scales 
measuring applicability and usefulness to hiring personnel. Panelists' comments were 
encouraged at each round to clarify large rating and ranking score variances, define 
qualities and characteristics of performance product ideas and comment on other data 
generated by the panelists. 

Findings include a rank ordering by importance to the organization of all of the 
SCANS skills and competencies. This ranking shows the human interactive skills and 
competencies as the most critical, the technological and resource management skills as 
the least critical. Findings also include a listing of performance product ideas to signal 
proficiency in a particular skill or competency. Performance product ideas include 
traditional products such as completed application forms, transcripts, letters of 
recommendation, lists of extra-curricular activities, resumes, etc. Non-traditional 
products include an array of written documents, videotaped presentations, electronic 
media documentation, portfolios and exhibitions. 

Recommendations include making the findings available to area high schools for 
emphasis in their curricula, and studying a method in which the high school can assist the 
student in presenting the traditional performance product data in a single coherent and 
succinct instrument. Further research is recommended to define the specific qualities and 
characteristics of the non-traditional performance products identified by the panel as 
useful, and to identify methods by which these can be successfully targeted throughout a 
high school’s curriculum and instruction. 
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Introduction 



Throughout America in recent years, business, labor, and educational leaders have 
been coming together to articulate educational goals that reflect this convergence. Many 
groups have undertaken the task of identifying the skills and competencies that are 
required of entry level workers (for example, the Future of the American Workforce 
Conference, 1994; National Council on Education Standards and Testing (NCEST), 

1992; National Education Standards and Improvement Council, 1994; National 
Educational Goals Panel, 1992; National Skills Standards Board, 1994; New Standards 
Applied Learning Framework Project (Resnick & Wirt, 1996); and the Secretary's 
Commission on Achieving Necessary Skills (SCANS) 1991. 

The identified skills and competencies are intended to afford entry-level workers 
personal and financial success in the workplace, and to become conscientious and 
contributing members of society. The SCANS (Secretary of Labor's Commission on 
Achieving Necessary Skills, 1991) skills and competencies have been accepted in general 
by both industry and education as reasonable, comprehensive, logical, and attainable 
(Resnick and Wirt, 1996). In the state of Arizona, for example, the Arizona Department 
of Education commissioned Standards Design Teams to specifically tailor skills and 
competencies to match the identified needs of the Arizona workplace. The SCANS 
report served as a basis for these studies, which culminated in the Arizona Workplace 
Skills Standards (Arizona Department of Education, March 1997). These identified skills 
and competencies serve as the target workplace skills and competencies criteria for the 
current Arizona Student Achievement Program (ASAP) which is intended to guide the 
curriculum throughout the state. 

The SCANS recommendations call for high school graduates entering the 
workforce to be capable of demonstrating performance objectives (SCANS: What Work 
Requires Of Schools, 1991). For example, entry-level workers should be able to 
participate as a member of a team, communicate ideas to justify a position both orally and 
in writing, acquire and evaluate information, perform basic computations and approach 
practical problems by choosing appropriately from a variety of mathematical techniques, 
and demonstrate understanding, friendliness, adaptability, empathy, and politeness in 
group settings (SCANS 1991). All of the SCANS recommendations are written 
specifically to target performance goals. 

In many high schools, however, learning is measured by primarily using 
standardized tests. The standardized test can, in fact, more easily assess the knowledge 
retention of students on a large scale because scores can be easily quantified and 
statistical analyses can be performed to derive findings about population groups. 
Standardized tests, however, cannot measure the actual performance capabilities of 
students (Berliner 1995, p. 318; Darling-Hammond, 1995, p. 2; Stiggins, 1997, p. 79; 
Wiggins, 1993, p. 15). 
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In order to align high school outcomes with workplace demands, performance 
outcomes and performance assessments seem logical (Stiggins, p. 79-88). This study did 
not present arguments of performance assessment versus standardized assessment. It 
proceeded under the assumption that both have legitimate educational purposes. The 
study began with the performance criteria established in the SCANS Report, which 
served as a basis for launching this study. 

The SCANS Report identified workplace skills and competencies on a national 
level. It did not accomplish this on the local level for each school district in America. 
Further, the SCANS reports did not establish performance assessments to measure 
individual proficiency in the identified skills and competencies. For educational 
practitioners at the local level, questions arise. Are the workforce skills and 
competencies desired by the local community different from the nationally identified 
skills and competencies? If so, how? What constitutes achievement of proficiency in 
each skill and competency? What assessments should be utilized to best measure 
proficiency? Who should make these determinations? 

It seems obvious that the local community workplace would answer the question 
of whether the locally desired workplace skills and competencies are different from the 
national ones. The community workplace should also identify the differences, if any. It 
is not so obvious who should answer the questions regarding what constitutes 
achievement of proficiency and what assessments should be used. The employers have a 
stake in determining the correct answer to these questions by virtue of the fact that they 
will be hiring these workers and will expect performance proficiency. The educational 
community has a stake in answering these questions because they, inevitably, are held 
accountable for providing the proper instruction to ensure proficiency. It is the author’s 
contention that this is an extremely critical area for communication between the 
education community and the employment community, an area where a major 
communications gap currently exists. It is precisely at this gap that both the education 
community and the employment community must again come together to answer the 
question, what constitutes achievement of proficiency and what assessments should be 
used? 

In order for performance achievement standards to be valid in the workplace, the 
workplace must have input into their establishment. It follows, therefore, that the 
workplace should also have input into the types of performance assessments used to 
measure proficiency. These performance assessments must be of sufficient detail and 
complexity to afford the employer a viable means of understanding the level of 
achievement that each entry-level employee has reached. What can the student have in 
his/her possession after graduation that will signal the employer that he/she has 
proficiency in the skills and competencies that the employer requires? Are these 
performance products to be in the traditional form of diplomas and transcripts, or in the 
nontraditional form of portfolios, photographs, videos, projects, endorsements, essays, 
and certifications, or combinations of these? 
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Adhering to the philosophy that American schools are accountable to their 
communities, the purpose of the study was to determine the specific SCANS skills and 
competencies that the city of Tempe, Arizona and vicinity employers look for in their 
entry-level employees and to identify specific student-produced performance products 
which employers would use to evaluate applicants in their hiring processes. 
Encompassing a variety of different industries and company sizes that employ entry-level 
high school graduates, and based on the assumption that most high school graduates will 
accept initial employment within their home communities, the findings were intended to 
be generalized back to the local community and its high schools. 

Closing the Skills Gap 

The skills gap demonstrates the difference between the quantity and quality of a 
worker's skills and the demands of his particular job. There are two schools of thought 
concerning this skills gap: first, that the pool of workers is less qualified than in previous 
times, and second, that job requirements increase over time and with innovation, but that 
worker's abilities do not increase along with them. Both are likely to be true. According 
to research by the National Assessment of Educational Progress (NAEP), academic 
performance in the U.S. has not fallen off dramatically since 1969. Rather, the demands 
of the external competitive environment have increased (Stone, 1991; Eck, 1993; Szabo, 

1 993). Other surveys contend that the youth graduating from our high schools do not 
possess even the most basic of skills (SCANS, 1991; Steck, 1992; Packer, 1993; ACT, 
Inc., 1998). Efforts to remedy the skills gap situation, therefore, also vary depending 
upon the point of view. Those who adhere to the "jobs are changing faster" theory 
address training in the workplace. Those who espouse the "poor graduate skills" theory 
address education in the schools. 

There are any number of strategies that schools are using to address the 
instruction and assessment of basic workplace skills. Also, based on the growth in the 
acceptance of performance-based assessment, there exists a growing body of literature 
and thought dealing with the manner in which high school graduates can signal 
employers that they do possess the basic skills that employers are looking for. 

Traditional diplomas and transcripts cannot convey the results of authentic performance 
assessments. Well-constructed performance-based assessments require a greater number 
of learner "thinking" and "action" steps than those responses required by typical multiple- 
choice standardized tests. These steps combine to produce a more complex and thorough 
performance product, which can provide a more useful insight into an individual’s 
proficiency level with regard to targeted skills and competencies. Traditional products 
that graduates use in signaling their achievements to potential employers have been 
diplomas, transcripts, resumes, and letters of reference from teachers, coaches, or 
administrators. Many other products are being discussed in the education community 
today: basic skills certificates and certificates of mastery (Tucker, 1996; Berkson, 1998), 
portfolios (Wiggins, 1993; Sizer, 1984; Berliner, 1995; Darling-Hammond, Ancess & 
Falk, 1995; Stiggins, 1997), digital portfolios (Wiedmer, 1998), statewide achievement 
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exams (Elmore, 1996; ASAP, 1997), Baccalaureate degrees (Bishop, 1996), competency 
profiles (Bishop, 1996). Also, commercial proprietary methods have been introduced 
such as ACT, Inc.'s WorkKeys ™ and ETS's Worklinks, as well as governmental 
innovations such as Toronto's (Canada) Benchmarks, Michigan's electronically accessed 
portfolio system, and California’s statewide portfolio system called the Career-Tech 
Assessment Project. 

Study Procedures 

This study employed a Delphi method of inquiry to generate data from a panel of 
twenty-four business and non-business employment professionals representing a broad 
range of industries and company sizes. According to Scheele (1975), a successful mix of 
respondents on a Delphi panel includes stakeholders, those who are or will be directly 
affected; experts, those who have an applicable specialty or relevant experience; and 
facilitators, those who have skills in clarifying, organizing, synthesizing, stimulating (p. 
68). Stakeholders, in the context of this study, would be recent high school graduates. 
Because these individuals would have no first hand experience about employment 
requirements in the study area, they were not included in the panel makeup. Table 1 
illustrates the characteristics of the panel of experts. 

Table 1 

Characteristics of Panel Members 



Occupational Category Organization Size 


Gender 


Number of Respondents 


Retail General Merchandise 


100+ 


F 


1 


Retail Apparel & Accessories 


50+ 


F 


1 


Retail Grocery 


50+ 


M 


1 


Retail Hardware Sales 


10+ 


F 


1 


Automobile Dealership 


50+ 


M 


1 


Full Service Restaurant Chain 


100+ 


M 


1 


Quick Service Restaurant Chain 


100+ 


F 


1 


Hotel/Resort 


100+ 


F 


1 


Property Management 


less than 10 


M 


1 


Telecommunications 


100+ 


F 


1 


Bank Credit Card Center 


100+ 


M 


1 


Temporary Personnel Service 


100+ 


M 


1 


Personnel Consulting 


less than 10 


M 


2 


Computer Distributor 


100+ 


F 


1 


City Government 


100+ 


M 


1 


Semiconductor Manufacturer 


100+ 


F 


2 


Industrial Electronics Manufacturer 


100+ 


F 


1 


Medical Devices Manufacturer 


100+ 


M 


1 


Computer Component Manufacturer 


100+ 


F 


1 


Electric Utility 


100+ 


F 


1 


High School Educator 


100+ 


F 


1 


Community College Educator 


100+ 


F 


_L 

24 




Development and Administration of the Delphi Questionnaires 
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In developing a list of entry-level workplace skills and competencies, the 
Secretary's Commission on Achieving Necessary Skills (SCANS) spent 12 months 
talking to business owners, public employers, managers, union officials and on the line 
workers in stores, offices, factories and government offices across the country. The 
findings of the commission were published in a series of publications beginning with 
What Work Requires of Schools: A SCANS Report for America 2000 (1991T Because 
it has wide acceptance in the business, labor, government, and education communities, it 
was used as the foundation for assessing the skills and competencies that employers in 
the study area wanted. In the present study, the Round One Questionnaire listed the 
complete definition of each skill and competency, one per page, as quoted directly from 
Skills and Tasks for Jobs: A SCANS Report for America 2000 . (SCANS, 1992, pp. 2-5 - 
- 2-9). Following each skill and competency definition on each page was a five-point 
Likert scale for the respondent to complete the initial rating. The Likert scale was 
presented as follows: not critical = 1, somewhat critical = 2, moderately critical = 3, 
highly critical = 4, and extremely critical = 5. After rating each SCANS skill and 
competency, each panelist was asked to generate a list of proof-of-performance products 
that they do use, or could use, to evaluate proficiency for each skill and competency. A 
list of over 125 performance products ideas were furnished to the panelists to stimulate 
ideas regarding possibly useful performance products that they could or would employ in 
their hiring evaluation practices. Panelists were not limited to the products on the list, but 
were encouraged to think of others as well. 

The Round Two Questionnaire was constructed using the results tabulated from 
the Round One Questionnaire. Each panel member could review his or her own rating 
responses from Round One in relation to the entire panels’ responses, and provide a 
second rating. Following the Likert rating scale was a listing of all the performance 
product ideas generated in the first round by the panel members for each skill and 
competency. Panel members then rated the applicability of each performance product 
idea to their own institution using another Likert scale rating system as follows: not 
applicable = 1, somewhat applicable = 2, moderately applicable = 3, highly applicable = 
4, extremely applicable = 5. The panel also ranked their top five choices from 5 to 1, 
with 5 being the most useful. 

The Round Three Questionnaire was constructed using the results tabulated from 
the Round Two Questionnaire. Panel members were shown summaries of the previous 
round’s ratings and rankings of both the individual SCANS skills and competencies and 
the proof-of-performance products. They then confirmed their final ratings of each 
SCANS competency and skill, and provided their final rating and ranking for each 
performance product idea generated. 

Within the methodologies employed in the Delphi technique, there is not an 
established standard for measuring significance, per se. The quantitative methods used in 
each Delphi application would determine its measurements of significance. Quantitative 
measurements employed in this study are the measurements of central location. 
Specifically, the mean was chosen as the measurement for comparing rating scores to 
ranking scores because it has the property that, for most distributions, it is a more 
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accurate and efficient estimate of the population mean than other measures of central 
location. It is subject to less error (Ferguson & Takane, 1989). 

In comparing the group mean score for each performance product rating to its 
respective ranking score, a value of 3.0 was assigned to the mean scores as the degree of 
significance. The rating score of 3.0 represented moderate applicability of a given 
performance product to a panel member’s organizational hiring practices. Assignment 
of the value of 3.0 for the degree of significance is a decision made by the researcher 
after the initial consultations with each panel member. It was determined that, although a 
subjective judgment, moderate applicability would have to obtain for innovation to be 
chanced. This is to say, employment personnel responsible for hiring entry-level workers 
would not be likely to engage in the evaluation of performance products presented by 
applicants if those products were not, at least, moderately applicable to that organization's 
skill and competency requirements. 

As depicted in the research graphic data, the mean rating scores for each 
performance product were plotted in descending order, highest to lowest, to obtain a 
depiction of the most applicable performance products for each SCANS skill and 
competency. See Figure 1 below, for an example of how the performance product data 
for each competency and skill were depicted. The mean ranking scores were plotted 
subordinate to the mean rating scores because all panelists rated each performance 
product, but not all panelists ranked each performance product. After rating each 
performance product, the panelists were asked to rank only the top 5 performance 
products in descending order of usefulness, 5 being the most useftil. Turoff (Linstone & 
Turoff, 1975) discusses the importance of taking care to avoid confusing and "painful" 
tasks for panel members (p. 84-101) which involve rank ordering large numbers of 
variables in a specific set. In this study, up to 20 performance product ideas were 
generated for a single SCANS skill. Asking a panel member to rank order 20 items, 
many of which may not have received a high rating by the that panelist, would have 
risked producing disinterested ratings resulting in biased and invalid data. 

Performance product mean ratings over the value of 3.0 accompanied by 
respective mean rankings over the value of 3.0 suggest the possibility of a relatively valid 
performance product. Standard deviations for the ratings and the rankings, as well as the 
frequency of ranking responses for each performance product were also displayed in an 
accompanying table to each graph. Although the correlation of the means above the 
value of 3.0 suggest significantly valid products, the ranking frequency responses and 
deviation values are a useful check. For example, where high mean ranking scores were 
shown in the data, but associated with low rating scores, the ranking response frequencies 
generally demonstrated that there were very few respondents who ranked the product. 

The validity of the resulting judgment of an entire group in a Delphi inquiry is 
typically measured in terms of the explicit "degree of consensus" among the experts 
(Linstone & Turoff, 1975, p. 22). Linstone & Turoff (1975) state: "The guarantor of such 
systems has traditionally been the function of human agreement, i.e., an empirical 
generalization (or communication) is judged 'objective,' or 'true,' or 'factual' if there is 
'sufficient widespread agreement' on it by a group of 'experts"' (p. 21). The degree of 
consensus is the measure of validity. 
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Figure 1 . Example of performance product data summary from research study. 




Mean Mean SP SP f F5 Listening 



Figure 5/Table 10 
Listening 

Performance Products 
by Rating and Ranking 



4.429 


4.429 


1.028 


1.028 


21 


1 


3.381 


3.308 


1.244 


1.437 


13 


2 


3.143 


2.786 


1.424 


1.578 


14 


3 


3.045 


2.889 


1.174 


1.269 


9 


4 


2.864 


2.750 


0.941 


1.138 


12 


5 


2.857 


2.250 


1.153 


1.165 


8 


6 


2.810 


2.400 


1.365 


0.966 


10 


7 


2.632 


3.750 


1.212 


0.957 


4 


8 


2.381 


1.333 


1.161 


0.577 


3 


9 


2.182 


3.000 


1.053 


1.414 


4 


10 


2.136 


2.000 


1.207 


0.926 


8 
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Observation of applicant during interview 

Summary multi-discipline senior year project 

Role-play video - project, performance, 

Standardized test results 

Letters of recommendation 

Report from school indicating competency 

Evaluation/critique of newsworthy speech 

Follow instructions on a VRU 

Diploma 

Dale Carnegie class 
DECA membership 



Figure 5/Table 10, Listening Performance Products by Rating and Ranking, 
above, introduces the importance of applicant performance in the interview, the event at 
which all panelists indicated they would evaluate listening proficiency. The summary 
multi-discipline senior year project is again valued highly by the panel; a third consistent 
high scoring of this performance product. Video documentation of a role-play project or 
performance is introduced as a viable alternative media. Standardized test scores again 
appear valid. Letters of recommendation appear valid, by virtue of the ranking response 
frequency of over half of the respondents (/= 12). 




RT = rating; RK = ranking; SD = standard deviation; / = frequency of responses; F5 Listening = skill 
identification 
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In this study, reliability was demonstrated by calculating a measure of interjudge 
agreement. According to Smith & Glass (1987): 

For those methods of measurement that require ratings, behavioral observations, 
or judgments of one individual by another, there are techniques for estimating the 
error associated with the observer. The procedure requires the same individual or 
event to be observed or rated by two or more judges. The ratings given by the 
judges are then correlated or the percentage of agreement calculated to provide a 
measure of inteijudge reliability. The resultant statistics reflect the ability to 
generalize across potential judges or observers. A high value of the statistic 
indicates that the quantities assigned to the individuals are not limited by the 
personality and abilities of one judge (104-105). 

The calculation of inteijudge agreement was undertaken at the completion of the 
first round of the Delphi study. The expert panel had completed one rating of the skills 
and competencies. The degree of variance of their ratings around the mean would be 
highest at this first rating point in the study.* A calculation of all ratings that fell outside 
± 1 standard deviation from the mean of each skill and competency showed a range of 
variation on each rated skill and competency from 0 to .38. The average overall degree 
of variation (disagreement) was .11969. This indicates that there was an overall 
inteijudge reliability degree of agreement of .88 or 88%. 

Findings 

SCANS Skills and Competencies Ratings 

Findings regarding the importance of the SCANS skills and competencies 
demonstrated that those skills dealing with human interactive abilities rated highest. 
Table 2, Ratings of SCANS Foundation Skills (F) and Competencies (C) Sorted by Mean 
Rating Value, shows that the panel believed Integrity/Honesty, Reading, Participating as 
Part of a Team, Responsibility, Works with Cultural Diversity, Serves Clients and 
Customers, Listening, Sociability, Speaking, Knowing How to Learn, Arithmetic, and 
Writing, rated in the upper third of all the skills and competencies in importance. 

Skills and competencies that fell in the middle range of importance were, Self 
Management, Problem Solving, Allocates Time, Self-Esteem, Decision Making, 
Reasoning, and Creative Thinking, Uses Computers to Process Information, 

Mathematics, Interprets/Communicates Information, Acquires & Evaluates Information, 
Organizes/Maintains Information, and Exercises Leadership. 

Although there was a significant degree of disagreement among panel members, 
overall, the comments reflected low demand for the remaining skills and competencies: 



* Helmer and Dalkey showed an interest in the observation that the variance of the responses around the 
median decreases with increasing number of rounds (Helmer, 1966). In the above calculation, the 
researcher chose the mean as the point of central measurement and a degree of variance equal to ± 1 
standard deviation around the mean as the estimation of the degree of consensus as inferred by Scheibe, 
Skutsch and Schofer (1975, p. 276). 
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Negotiates to Arrive at a Decision, Understands Systems, Teaches Others, Improves or 
Designs Systems, Selects Technology, Applies Technology to Tasks, Allocates Human 
Resources, Monitors/Corrects Performance, Allocates Money, Seeing In the Mind’s-Eye, 
Maintains/Troubleshoots Equipment, and Allocates Materials & Facilities. 

Table 2 



Ratings of SCANS Foundation Skills (F)and Competencies (C) Sorted by Mean Rating Value 



Code 


Skill / Competency 


Mean 


Std dev 


F17 


Integrity/Honesty 


4.850 


0.366 


FI 


Reading 


4.750 


0.444 


C9 


Participates a Part of a Team 


4.700 


0.470 


F13 


Responsibility 


4.650 


0.489 


C14 


Works with Cultural Diversity 


4.650 


0.671 


Cll 


Serves Clients & Customers 


4.600 


0.598 


F5 


Listening 


4.550 


0.510 


F15 


Sociability 


4.158 


0.688 


F6 


Speaking 


4.150 


0.671 


FI 1 


Knowing How to Learn 


4.100 


0.718 


F3 


Arithmetic 


4.000 


0.725 


F2 


Writing 


3.950 


0.605 


F16 


Self Management 


3.947 


1.026 


F9 


Problem Solving 


3.900 


0.718 


Cl 


Allocates Time 


3.850 


0.745 


F14 


Self-Esteem 


3.750 


0.910 


F8 


Decision Making 


3.650 


0.587 


F12 


Reasoning 


3.600 


0.883 


F7 


Creative Thinking 


3.550 


0.826 


C8 


Uses Computers to Process Information 


3.450 


0.999 


F4 


Mathematics 


3.300 


0.865 


C7 


Interprets/Communicates Information 


3.300 


0.571 


C5 


Acquires & Evaluates Information 


3.150 


0.447 


C6 


Organizes/Maintains Information 


3.150 


0.489 


C12 


Exercises Leadership 


3.050 


0.686 


C13 


Negotiates to Arrive at a Decision 


3.050 


0.686 


C15 


Understands Systems 


3.050 


0.686 


CIO 


Teaches Others 


2.950 


0.999 


C17 


Improves or Designs Systems 


2.450 


1.191 


C18 


Selects Technology 


2.400 


0.883 


C19 


Applies Technology to Tasks 


2.400 


0.995 


C4 


Allocates Human Resources 


2.300 


0.979 


C16 


Monitors/Corrects Performance 


2.300 


0.979 


C2 


Allocates Money 


2.263 


0.933 


F10 


Seeing-Mind's Eye 


2.250 


0.851 


C20 


Maintains/Troubleshoots Equipment 


2.200 


0.951 


C3 


Allocates Materials & Facilities 


2.100 


0.718 



n=20, missing 4 (from total of 24 panelists), maximum mean rating = 5 
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In a similar study conducted with a panel of small businesses throughout the state 
of Oklahoma, very similar results were found (Harris, 1996). In that study, using similar 
rating methods, 9 out of the top 12 skills and competencies found in the present study 
were also found. Integrity/Honesty topped the list in both studies. Among the lowest 
rated 12 skills and competencies in the present study, fully 8 of the same skills and 
competencies were found in both studies. 

Proof-of-Performance Product Ratings and Rankings 

This study identified many proof-of-performance product ideas. The panel 
generated a list of traditional and nontraditional proof-of-performance products for each 
of the 37 SCANS skills and competencies. Those performance products that employers 
traditionally use to evaluate entry-level job applicants were listed and discussed in some 
detail. For example, employer testing, letters of recommendation, past employer 
evaluations, references, resumes, and transcripts were mentioned frequently. The 
traditional interview process itself is also a critical event at which employment personnel 
assess the skill and competency levels of prospective employees. Many of the panelists 
said they focus on non-tangible products which applicants demonstrate during an 
interview through conversation, verbal responses to specific questions, gestures, and 
body language. 

A number of nontraditional performance product ideas were also generated in the 
study. Several panel members suggested the use of various media to document student 
proficiency in selected skill areas. For example, video taped documentation was 
suggested for demonstrating proficiency in many areas including listening, speaking, 
creative thinking, reasoning, and self management, among others. The use of electronic 
media, computer diskettes, and the construction of internet web pages was suggested for 
use in demonstrating proficiency in the areas of creative thinking, organizing and 
maintaining information, interpreting and communicating information, and processing 
information. A comprehensive senior year research project received high ratings from 
the panel. The senior project, described much like an exhibition for graduation promoted 
by the Coalition of Essential Schools (Sizer, 1992), is intended to demonstrate 
proficiency in higher order thinking across a majority of the SCANS skills and 
competencies in a single comprehensive project. Letters, essays and calculations were 
suggested for certain skills and competencies. Standardized test scores, not a traditional 
employer tool for measuring applicant proficiency, were suggested several times to assess 
basic skills. Also, there were several product ideas suggested that fall outside the 
parameters of the traditional high school curriculum, such as scouting awards and Dale 
Carnegie courses. 
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Conclusions 
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• The highest valued skills and competencies by employers require an employee to 
interact with, and relate well with others in the workplace. These skills and 
competencies were, in order of their respective mean rating scores, Integrity/Honesty, 
Reading, Participating as Part of a Team, Responsibility, Works with Cultural 
Diversity, Serves Clients and Customers, and Listening. 

• Somewhat contrary to much educational discourse today, the least important skills 
and competencies were those calling for knowledge of technology and resource 
management. Based on panelists' comments and ratings of these skills and 
competencies as non-critical, it can be concluded that these would be less important 
in employers' assessment procedures of entry-level workers. These skills and 
competencies are Improves or Designs Systems, Selects Technology, Applies 
Technology to Tasks, Allocates Human Resources, Monitors and Corrects 
Performance, Allocates Money, Sees Things in the Mind's Eye, Maintains and 
Troubleshoots Equipment, and Allocates Materials and Facility Resources. 

• Traditional proof-of-performance products will continue to be used by hiring 
personnel. These include neat and complete applications, employer testing and 
interviewing, resumes, transcripts, references, letters of recommendation, and lists of 
certifications, awards and honors. 

• Non-traditional proof-of-performance products directly demonstrating higher-order 
skills within the performance product itself include a variety of written documents, 
videotaped activities, and electronic (computer-related) media. These include the use 
of exhibitions, documented research, electronic portfolios, written essays, self 
reflected analyses and published articles, role-plays and speeches, marketing or 
business plans, brochures, pamphlets and newsletters, goal setting and personal 
improvement plans, work flow evaluations, cause and effect sequences, group 
decision-making documentation, attendance and tardy reports, grade level progression 
and on-time graduation, personal budgets and savings plans. 

• Based on the comparative ratings and rankings of the performance product ideas 
generated by the panel members, both traditional and nontraditional types of 
performance products are worthwhile and should be constructed by students while 
still in school. More study is needed in the area of the nontraditional performance 
products to further define the specific qualities and characteristics of these products 
that directly demonstrate higher-order skills and signal performance proficiency to 
employers. 
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Recommendations 
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• Based on the conclusions regarding the importance of the respective SCANS skills 
and competencies, area high schools should implement strategies for augmenting the 
curriculum and instruction to teach these specific skills and competencies in the 
classrooms. Given the non-technical nature of the highest rated skills and 
competencies dealing with human interaction, they should be reinforced throughout 
all of the disciplines, wherever possible. 

• Within the parameters of the Family Educational Right to Privacy Act (Buckley 
Amendment), area high school administrators, teachers and students should study 
methods in which they can develop a single instrument that adequately and succinctly 
provides prospective employers all of the traditional performance data identified in 
this research. These data include standardized and other relevant test scores, 
attendance records, transcripts, achievements and awards, clubs and extra-curricular 
activities, and references, testimonials or letters of recommendation from 
administrators, teachers, and coaches. 

• It is recommended that further research of the nontraditional proof-of-performance 
product ideas generated by this study be carried out. Specifically, this research 
should further define the qualities and characteristics of the nontraditional proof-of- 
performance products rated highest for signaling proficiency in each respective 
SCANS skill and competency. 

« Following the research suggested above, a pilot program/research study in an area 
high school, or school-within-a-school, should be established to specifically target the 
curriculum and instruction to fully implement the recommendations of the research. 
This program should include a longitudinal cohort study to follow workers who 
graduated from the program throughout several years of employment after 
graduation. 
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